Integrating Surprisal and Dependency Locality Theory: A Broad Coverage Model

Dependency Locality Theory (DLT; Gibson 1998) predicts processing difficulty at the head of a phrase based on the notion of integration cost (IC), a distancebased measure of the processing effort required when the head is integrated with its syntactic dependents. An alternative account of processing difficulty is Surprisal (Hale, 2001), where the Surprisal of a word corresponds to the amount of information that has to be processed when encountering this word. Surprisal is estimated using a probabilistic grammar and has been shown to predict a variety of experimental results (Levy, 2008). Recently, Demberg & Keller (2007) tested both IC and Surprisal on an eyetracking corpus, and found that Surprisal, but not IC, predicts reading times in a broad-coverage setting. Here, we propose a variant of DLT capable of making broad coverage predictions. We redefine integration cost in terms of surprisal. At a head wh, the surprisal-based integration cost (SIC) of a syntactic dependent wd is defined as Sd...h, the cumulative surprisal of the words between the dependent and its head. If d and h are adjacent, then this corresponds to standard surprisal, but if there are intervening words, we need the surprisal of a region of words, which is defined straightforwardly as Sd...h = logP(w1 · · ·wh)− logP(w1 · · ·wd). The total SIC at h is the sum of Sd...h over all dependents of h. SIC is distance-based like standard IC, but distance is measured as the surprisal of the words intervening between d and h, not as the number of intervening discourse referents as in standard IC. Cognitively, this corresponds to the assumption that intervening high-surprisal material makes it more difficult to keep a dependent in memory (due to increased memory load). We evaluated SIC on the Dundee Corpus (Kennedy & Pynte, 2005), which contains the eyetracking record of 10 subjects reading 51,000 words of newspaper text. We fitted a hierarchical mixed effects model that included reading time as the dependent variable and either SIC, standard IC, or Surprisal as the target variable. The model also included nine control variables known to influence reading times, both linguistic ones (such as word frequency) and eye-movement ones (such as launch distance). Surprisal values were computed using Roark’s (2001) incremental lexicalized parser. We tested the resulting models on the verbs and nouns in the corpus (the integration cost is zero for all other words), and found that Surprisal was not a signif-